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ABSTRACT 


The source selection decision in a competitively 
negotiated acquisition is very difficult. Cost realism is 
one evaluation factor that may be used to assist in making 
the source selection decision. In this study the concept 
of cost realism and feasible methodologies for achieving it 
are investigated. The role of cost realism in the source 
selection plan and the source selection process is 
identified and defined. Specific cost realism issues are 
reviewed and discussed. Cost realism utilization in the 
Department of the Navy is investigated. The study 
concludes that cost realism provides useful assistance in 


making the source selection decision. 
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1. INTRODUCTION 


A. GENERAL 

The source selection decision in a competitively 
negotiated acquisition that will result in award of a cost 
reimbursable contract is very difficult. The contracting 
officer or designated source selection authority must: 

-.-ensure selection of the source whose proposal has the 
highest degree of realism and whose performance is 
expected to best meet stated government requirements. 
[Ref. l:p. 15-19] 

The selection of the proposal with the highest degree 
of realism and performance is judgemental on the part of 
the contracting officer or source selection authority. The 
selection is based on a careful evaluation of the factors 
that make up the proposal. Additionally, the Federal 
Acquisition Regulation (FAR) identifies other evaluation 
factors that may be considered which include cost realism, 
technical excellence, management capability, personnel 
qualifications, experience, past performance, schedule and 
other relevant factors. 

In procuring products or services, the preferred method 
is to issue an Invitation for Bid (IFB) that completely 
identifies the product or service desired. A Firm Fixed 


Price (FFP) contract award is then made to the offeror of 


the bid that offers the lowest cost. This method of 


procurement works best for products and services for which 
performance and/or design specifications are firm and more 
than one supplier is willing and capable of providing the 
product or service. When design or performance 
specifications are not firm, as in the case of research and 
development or engineering services procurements, there is 
a high degree of performance, schedule and cost risk. 

To compensate for these risks ina fixed price contract 
environment, the contractor will include costs for 
additional labor, material, overhead and profit. If the 
additional cost items are not utilized during contract 
performance, the contractor keeps the additional costs as 
increased profit. If the contractor is unable to deliver 
the required product or services within the contract price, 
the contractor will suffer a loss that may be severe enough 
to force the contractor to default on the contract. Both 
of these situations, excess profit or termination for 
default, are considered unacceptable to the government. 

For procurements with risk levels that are considered 
unacceptable for fixed price contracts, the government 
assumes the risk by awarding a cost reimbursable contract. 
This type of contract requires the government to reimburse 
all of the contractor's allowable and allocable costs 
incurred during contract performance. Choosing a contract 
source for a cost reimbursable contract award is not as 


Simple and easy as for a fixed price contract award. To 
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make a contract award in this situation based entirely upon 
the lowest proposed cost would be unrealistic, since the 
final cost of performance may bear no resemblance to the 
original proposed cost. Additionally, the award of a cost 
reimbursable contract obligates the government to reimburse 
the contractor's allowable and allocable costs. Large 
increases in cost during a contract's performance seriously 
undermine both the Congress' and public's faith in the Navy 
and significantly impairs the Navy's planning, programming 
and budgeting system. The use of cost realism as an 
evaluation factor in the source selection process could 
provide some degree of assurance that proposed costs are 


realistic and defensible. 


B. OBJECTIVE OF THE RESEARCH 

The objective of this study is to explore how the 
source selection process evaluates contractor proposals for 
cost realism and how it utilizes this evaluation in source 


selection. 


C. RESEARCH QUESTIONS 
To achieve the objective of the research the following 
question was posed: 


Can the cost realism concept be used to improve the 
source selection process? 


To answer the basic research question, the following 
subsidiary questions were asked: 


1. What is cost realism? 
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2. At what points and under what circumstances may cost 
realism become an issue in the source selection 
process? 

3. How can cost realism best be utilized in the source 
selection process? 

D. RESEARCH METHODOLOGY 

The information presented in this research was obtained 
from currently available procurement related literature and 
personal interviews with contracting officers and program 
management office personnel. Literature was obtained from 
the Naval Postgraduate School Library, Defense Technical 
Information Center, and Defense Logistics Studies 
Information Exchange. Applicable regulations, directives, 
and instructions that govern the acquisition process were 


utilized. Personnel interviewed during the research effort 


are identified in the List of References. 


E. SCOPE AND LIMITATIONS 

This research is limited to cost realism issues and 
policies associated with major weapons systems acquisitions 
in the Department of the Navy (DON). The thrust of the 
research was on developing an understanding of the cost 
realism issues present in the source selection process and 
how these issues are recognized and dealt with by the 
contracting officer, Source Selection Evaluation Board 
(SSEB), Source Selection Advisory Council (SSAC), and 


Source Selection Authority (SSA). 
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F. ORGANIZATION OF THE STUDY 

This thesis is organized to provide the reader with an 
examination of the cost realism concept and a basic 
understanding of its role in the source selection 
decision. The thesis is divided into the following 
chapters: 

1. Chapter II introduces the concept of cost realism 
through a discussion of the source selection environment 
and the source selection process leading up to the 
definition of cost realism. 

2. Chapter III reviews and discusses where and when 
cost realism issues are specifically addressed during the 
source selection process. 

3. Chapter IV discusses how cost realism is utilized 
in the Department of the Navy. 

4. Chapter V summarizes the results of the research, 
provides conclusions and recommendations developed as a 
result of this study and provides recommendations for 


further study. 
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II. THE COST REALISM CONCEPT IN PERSPECTIVE 


A. INTRODUCTION 

This chapter examines the concept of cost realism and 
brings it into perspective. The framework for 
investigation of cost realism is initiated by a discussion 
of the source selection environment and the source 
selection process. Next, the roles of cost and price in 
the source selection process are discussed. Finally, the 
cost realism concept is defined, and considerations are 
presented to demonstrate its position and role within the 


overall process. 


B. THE SOURCE SELECTION ENVIRONMENT 

Major weapon systems are acquired by the Department of 
the Navy to provide hardware resources to operational 
forces enabling them to achieve operational objectives in 
their support of national policies and objectives. 

Major weapons systems are developed to fulfill a 
mission need. Mission need could be the result of advances 
in technology that render a current weapon system obsolete 
or advancement of enemy capabilities beyond current 
defensive capabilities. No matter what the reason for 
development of a major weapon system, several similarities 


are present in almost all new major weapon systems 
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acquisitions. First, the weapon system begins as a concept 
that utilizes the latest in technological advances and in 
some cases, utilizes technology that is still under 
development. Second, successful research, development, 
test and evaluation (RDT&E) of a technologically advanced 
major weapon system can involve many unkown factors and 
their attendant risks. Third, the government must carry 
most, if not all, of the cost risk associated with major 
weapon system development through the award of a cost 
reimbursable contract. 

The cost of a major weapon system is often measured in 
hundreds of millions or billions of dollars. When there is 
more than one source available to develop and manufacture 
the major weapon system, the resulting competition may be 
fierce and highly competitive. There is usually much more 
at stake than the profit and loss statement of a single 
contractor. Thousands of jobs could be at risk, not only 
with the primary contractor, but with sub-contractors and 
other "trickle-down economy" business in the surrounding 
communities. Additionally, local governments, special 
interest groups and congressional interests are always 
aroused whenever a government action has the potential for 
major impact on the economy, environment or votes. 

There is a large amount of pressure involved in the 
acquisition of a major weapon system. There is pressure on 


the program manager to develop an affordable, reliable and 
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technologically advanced system on time and under budget. 
There is pressure on the contractors to win the system 
award competition and keep profits and jobs secure. There 
is pressure on the contracting officer or source selection 
authority to procure the system from responsible sources at 
fair and reasonable prices calculated to result in the 
lowest ultimate cost to the government. 

The very real result of this pressure, when coupled 
with the technical, schedule and cost risks involved, could 
be the award and development of a major weapon system based 


on an unrealistic proposal. 


C. THE SOURCE SELECTION PROCESS 
Source selection is the process wherein the 
requirements, facts, recommendations and government 
policies relevant to an award decision in a competitive 
procurement of a system are examined, and the source 
selection decision is made. Source selection procedures 
are designed to: 
(a) maximize competition 
(b) minimize the complexity of the solicitation, 
evaluation, and the selection decision 
(c) ensure impartial and comprehensive evaluation of 
offerors' proposals and 
(d) ensure selection of the source whose proposal has the 
highest degree of realism and whose performance is 
expected to best meet stated Government requirements. 
[Ref. l:p. 15-19] 
These procedures are not only good business, but are 


also in keeping with the responsibilities inherent within 


the stewardship of public monies and trust. 
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The determination of those factors and attributes which 
identify a responsible source, as well as how a fair and 
reasonable price can be accurately calculated, has been, 
and remains today, a difficult problem for DoD to resolve. 
Deputy Secretary of Defense Carlucci recognized this 
problem when he reported that: 

Some DoD competitively-selected contractors have 
performed poorly. In some instances, source selection 
criteria do not sufficiently take into account past 
performance or plans for future phases of a program. 
Also, the credibility and realism of contractor cost 
proposals are not always challenged. [Ref. 2:p. 38] 

Failures in the DoD source selection process can easily 
result in unanticipated program cost growth which can then 
lead to budgetary shortfalls or schedule slippage due to 
inadequate funding. Additionally, when it becomes 
necessary to fix the blame, which it almost invariably 
does, the resultant finger pointing, accusations, 
counter-accusations, congressional hearings, Inspector 
General and General Accounting Office investigations and 
audits can result in either program redirection, reduction 
or cancellation. There is also the very real possibility 
for additional statutory or regulatory guidance being added 
to the already overwhelming and overburdened acquisition 
system. 


The Department of Defense recognized the problems 


associated with inadequate source selection decisions when 
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it issued Department of Defense Directive (DODD) 4105.62, 


Selection of Contractual Sources for Major Defense Systems, 


establishing detailed guidelines for the management of the 
source selection process. 

As part of the policy which governs the solicitation, 
evaluation and source selection process, DODD 4105.62 
states: 


1. The prime objectives of the process are to (a) 
select the source whose proposal has the highest degree 
of realism and credibility and whose performance is 
expected to best meet Government objectives at an 
affordable cost; (b) assure impartial, equitable, and 
comprehensive evaluation of competitors' proposals and 
related capabilities; and (c) maximize efficiency and 
minimize complexity of solicitation, evaluation and the 
selection decision. 


2. Each DoD component shall develop, and 
consistently apply, procedures which create the 
environment for an impartial, balanced and realistic 
appraisal of all proposals submitted. 

The prime objectives of the source selection process 
are therefore in keeping with Defense Acquisition 
Regulations (DAR) policy to procure from responsible 
sources at fair and reasonable prices. 

The source selection process as currently established 
requires an evaluation of the contractor's proposal and the 
capability of the contractor to successfully accomplish the 
program under consideration. 

The primary goal of evaluation is to determine, for 
each competitor, the most likely outcome in terms of 
system characteristics, costs (development, production, 
and support), and schedules....The evaluation should give 
consideration to each industrial competitor's assessment 
of the technical and financial risks involved and the 
credibility of proposed approaches to eliminate, avoid or 


minimize those risks.... [Ref. 3:p. 6] 
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This evaluation process is conducted in four steps: 


Step 1. Separate technical proposals shall be solicited 
and evaluated.... 


Step 2. A cost/price proposal shall then be 
obtained...together with any necessary revisions to correct 
deficiencies in the technical proposals...the competitive range 
will be established...meaningful discussions will be held with 
the remaining offerors.... 


Step 3. Following such discussions, a common cutoff date 
for the receipt of final revisions to technical and cost/price 
submittals will be established...after receipt of any revised 
submittals, the proposals shall be evaluated based upon the 
offeror's total proposal and a contractor selected for 
negotiation of the contract. 


Step 4. A definitive contract will then be negotiated with 

the selected offeror. [Ref. 3:p. 7] 

D. THE ROLES OF COST AND PRICE IN THE SOURCE SELECTION 

PROCESS 

1. Definitions 

In the source selection process, the determination 

of estimates for cost and price for a weapon system may 
become a point where the government and contractor take 
divergent courses. It becomes necessary, therefore, to 
define the terms frequently used in developing the 
estimates for cost and price: 

a. "Cost Analysis" is the review and evaluation of the 
separate cost elements and proposed profit of (a) an 
offeror's or contractor's cost or pricing data and 
(b) the judgmental factors applied in projecting from 
the data to the estimated costs, in order to form an 
opinion on the degree to which the proposed costs 
represent what the contract should cost, assuming 
reasonable economy and efficiency. [Ref. l:p. 15-25] 

b. "Cost or Pricing data" means all facts as of the time 


of price agreement that prudent buyers and sellers 
would reasonably expect to affect price negotiations 
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significantly. Cost or Pricing data are factual, not 
judgmental, and are therefore verifiable. While they 
do not indicate the accuracy of the prospective 
contractor's judgment about estimated future costs or 
projections, they do include the data forming the 
basis for that judgment. Cost or Pricing data are 
more than historical accounting data; they are all 
the facts that can be reasonably expected to 
contribute to the soundness of estimates of future 
costs and to the validity of determinations of costs 
already incurred. They also include such factors as 
(a) vendor quotations; (b) nonrecurring costs; (c) 
information on changes in production methods and in 
production or purchasing volume; (d) data supporting 
projections of business prospects and objectives and 
related operations costs; (e) unit-cost trends such 
as those associated with labor efficiency; (f) 
make-or-buy decisions; (g) estimated resources to 
attain business goals; and (h) information on 
management decisions that could have a significant 
bearing on costs. [Ref. l:p. 15-25] 


"Price" is the cost plus any fee or profit applicable 
to the contract type. [Ref. l:p. 15-26] 


"Price Analysis" is the process of examining and 
evaluating a proposed price without evaluating its 
separate cost elements and proposed profit. [Ref. l1: 
p. 15-26] 


"Should-Cost Analysis" is a specialized form of cost 
analysis employing an integrated team of government 
contracting, contract administration, pricing, audit, 
and engineering representatives. It differs from 
regular cost analysis in its depth, in the fact that 
it is conducted at the contractor's plant, and in the 
extent to which the Government identifies and 
challenges inefficiencies in the contractor's 
management and operations rather than merely 
challenging certain proposed costs. The purpose of 
should-cost analysis is to (1) identify uneconomical 
or inefficient practices in the contractor's 
management and operations, (2) quantify their impact 
on cost in order to develop a realistic price 
objective for negotiation, and (3) lead to both short 
and long-range improvements in the contractor's 
economy and efficiency. [Ref. l:p. 15-39] 


"Proposal Evaluation" is an assessment of both the 
proposal and the offeror's ability (as conveyed by 
the proposal) to successfully accomplish the 
prospective contract. [Ref. l:p. 15-21] 
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2. The Roles 

Before a weapon system can reach the source 
selection stage, the estimated cost of the weapon system 
has played a key role in the planning, programming and 
budgeting process. There is the strong possibility that 
Congressional authorization and appropriation for the 
weapon system was based on the estimated cost as much as it 
was based on the need for the systems' capabilities. 

The program manager must ensure that only realistic 
cost and budget information is provided and that such 
information is accurate and complete. Cost estimates must 
include the full anticipated development, production and 
operational costs associated with the program, even though 
this task is especially difficult at program initiation. 
The estimates support budgeting and early funding for 
testing, budgeting for most likely costs, budgeting for 
technical and scheduling risks, capitalization of 
production and specialized operational facilities and 
independent cost analysis. [Ref. 4:p. 1-9] 

FAR requires price or cost to the government be 
included as an evaluation factor in every source 
selection. Other evaluation factors previously identified 
that may apply to a particular acquisition are cost 
realism, technical excellence, management capability, 


personnel qualifications, experience, past performance, 
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schedule and other relevant factors. The estimated cost of 
the contractor's proposal will certainly be a factor in the 
evaluation process. Cost is a driving factor throughout 
the system acquisition process, but it must be put in 


perspective. 


In the development stage, the prime concern is to 
find and engage contractors who have the conceptual 
ideas, manpower, management expertise, facilities, and 
the demonstrated experience to develop a system capable 
of meeting the mission need. Cost estimates in the 
earlier stages of the acquisition process are far from 
precise, and independent estimates of development and 
Life Cycle Costs (LCC) by an inhouse activity are needed 
to establish a baseline against which to evaluate the 
validity of contractor cost estimates. With due regard 
for its significance, cost is a controllable element 
which can be managed through carefully drawn, properly 
executed contracts and through liberal use of competition 
throughout the acquisition process. The cost to develop 
a system is only one part, albeit an important one, of 
the system LCC. Costs incurred in development can return 
large dividends in the form of lower production and 
maintenance costs as well as in improved performance. 
[Ref. 4:p. 4-40] 


While the lowest price or lowest total cost to the 
government is properly the deciding factor in many source 
selections, in certain acquisitions the government may 
select the source whose proposal offers the greatest value 
to the government in terms of performance and other 


factors. 


In awarding a cost-reimbursement contract, the cost 
proposal should not be controlling, since advance 
estimates of cost may not be valid indicators for final 
actual costs. There is no requirement that 
cost-reimbursement contracts be awarded on the basis of 
lowest proposed cost, lowest proposed fee or the lowest 
total proposed cost plus fee. The award of 
cost-reimbursement contracts primarily on the basis of 
estimated costs may encourage the submission of 
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unrealistically low estimates and increase the likelihood 

of cost overruns. [Ref. l:p. 15-20] 
E. THE COST REALISM CONCEPT 

1. General 
Cost is a key factor in the source selection 
process. It may not be the most important factor, but its 
importance cannot be overlooked. The determination of a 
weapon systems' estimated cost through cost analysis is a 
judgemental decision. The decision may be influenced by 
optimism or pessimism toward any one or more of the factors 
which go into making up the total estimated cost. 
Traditionally, contractors have the tendency to lean toward 
optimism in cost estimation and the government has the 
tendency to lean toward pessimism. However, this is not 
always the rule, and situations do occur where the opposite 
may happen. Therefore, the methodology by which a 
contractor's cost proposal is evaluated to determine 
whether it is reasonable, realistic, credible and 
defensible becomes an important issue. 
2. Cost Realism Defined 
Cost realism may be defined as the methodology by 

which a contractor's cost proposal is evaluated during the 
source selection process to determine with some degree of 
accuracy and fairness whether or not it is a realistic 


estimate of what the actual costs are expected to be, 
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assuming no change in the government's requirements or the 
contractor's technical approach. 
3. Considerations 

FAR requires price or cost be included as an 
evaluation factor in every source selection, but an 
optional cost realism evaluation may be made at the 
discretion of agency acquisition officials. There is no 
definitive guidance in statutes or regulations for the 
conduct of a cost realism evaluation; however, there are a 
variety of cost analysis elements and techniques that may 
be used to determine the realism of proposed costs. When 
cost realism is included as an evaluation factor, the 
informed judgement rule requires the contracting officer or 
source selection authority make an analysis of proposed 
costs sufficient to make a supportable judgment that the 


costs proposed are realistic. 


F. SUMMARY 

This chapter provided a broad overview of the source 
selection environment and process for major weapon system 
acquisition. The complexity and cost of a major weapon 
system and the resultant economic impact on competition 
winners and losers makes the source selection decision very 
important. 

The evaluation of a proposal to determine its degree of 
realism requires a thorough analysis of every element which 


makes up the proposal. Cost is a required evaluation 
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factor and may become the controlling factor; therefore, a 
determination of the realism of proposed costs becomes a 
required evaluation factor. 

By this analogy, an evaluation of cost realism becomes 
required; however, its methodology is not defined in 
statute or regulation. This chapter presented a definition 
of cost realism as an evaluation factor in the source 


selection decision process. 
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III. COST REALISM ISSUES DURING SOURCE SELECTION 


A. INTRODUCTION 

It is the purpose of this chapter to review and discuss 
where and when cost realism issues are specifically 
addressed during the source selection process. For 
purposes of discussion, the chapter divides the source 
selection process into three parts. Part 1 encompasses all 
steps leading up to the release of the solicitation. Part 2 
covers proposal evaluation. Part 3 addresses the source 
selection decision. Additionally, this chapter briefly 


addresses the issue of cost realism based protests. 


B. PLANNING FOR COST REALISM 
1. The Source Selection Plan Groundwork 

When an acquisition objective has been established, 
an acquisition plan is prepared that addresses the 
acquisition process which will be followed to achieve the 
desired objective. 

A key part of the acquisition planning process is 
the source selection plan. The source selection plan, as a 


minimum, includes: 


(1) A description of the organization structure 

(2) Proposed presolicitation activities 

(3) A summary of the acquisition strategy 

(4) A statement of the proposed evaluation factors and 


their relative importance 
(5) A description of the evaluation process, methodology 
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and techniques to be used and 
(6) A schedule of significant milestones. [Ref. l:p. 15-23] 


2. The Solicitation 

The solicitation is the method by which the 
government's requirements are conveyed to potential 
contractors either as a Request for Proposal (RFP) or as a 
Request for Quotation (RFQ). 

Solicitations shall contain the information necessary to 
enable prospective contractors to prepare proposals.... 
[Ref. l:p. 15-10] 

The solicitation is prepared using the uniform 
contract format which facilitates preparation of both the 
solicitation and the contract. The principal difference 
between the solicitation and the contract is the inclusion 
of Part IV, Representations and Instructions, in the 
Solicitation. Part IV contains section K, Representations, 
Certifications and Other Statements of Offerors or Quoters, 
Section L, Instructions, Conditions and Notices to Offerors 
or Quoters and Section M, Evaluation Factors for Award. It 
is in Part IV, Sections L and M, that cost realism issues 
are first addressed. 

Section L contains: 

...-solicitation provisions and other information and 
instructions not required elsewhere to guide offerors or 
quoters in preparing proposals or quotations. [Ref. l:p. 
15-12] 

It is in this section that detailed guidance is 


provided for the submission of technical and cost or 


pricing data. The level of detail required is that which 
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is considered necessary for complete technical and cost 
analysis of the proposal. This routinely requires 
submission of technical and cost data in Work Breakdown 
Structure (WBS) format. In this way, cost and technical 
comparisons can be made between different offerors for the 
same WBS elements, and greater cost visiblity can be 
provided where considered necessary. 
Section M, of the solicitation: 
..-Shall clearly state the evaluation factors, including 
price or cost and any significant subfactors, that will 
be considered in making the source selection and their 
relative importance. Numerical weights, which may be 
employed in the evaluation of proposals, need not be 
disclosed in solicitation. [Ref. l:p. 15-20] 
However, it is necessary to: 
...state the relative importance the government places on 
those evalutation factors and subfactors. [{Ref. l:p. 
iby %4 | 
This is performed by identifying minimum 


requirements that apply to particular evaluation factors 


and significant subfactors. 


C. THE PROPOSAL EVALUATION 
Proposal evaluation is an assessment of both the 
proposal and offeror's ability (as conveyed by the 
proposal) to successfully accomplish the prospective 
contract. An agency shall evaluate competitive proposals 
solely on the factors specified in the solicitation. 
[Ref. lz:p. 15-21] 
The proposal evaluation factors normally specified 
in the solicitation are technical, cost and management 


capability. Other factors included for evaluation 
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purposes may include personnel qualifications, past 
performance, reliability and maintainability and schedule. 
For cost realism purposes, the primary evaluation factors 
which require careful and complete analysis are technical 
and cost. 

1. Technical Analysis 

The purpose of the technical analysis is to assess 
the offeror's ability to accomplish the minimum technical 
requirements. The analysis looks at the types and 
quantities of labor proposed, bills of material, proposed 
sub-contracts, proposed technical approach, assumptions of 
technical risk and other pertinent technical elements. 
Based on the results of the technical analysis, a summary, 
matrix or quantitative ranking of each technical proposal 
can be made as specified in the source selection plan. 

2. Cost Analysis 

Methodologies for conducting cost analysis may 
differ depending upon what is being procured and who is 
conducting the cost analysis. However, the basic goal of 
cost analysis remains the formation of: 

--an opinion on the degree to which the proposed costs 
represent what the contract should cost, assuming 
reasonable economy and efficiency. [Ref. l:p. 15-25] 

The government's opinion on costs usually takes the 
form of a government estimate of the most likely cost which 
will be incurred by the offeror in fulfilling the terms and 
conditions of the contract. 
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3. Overall Analysis 


While conducting technical and cost analyses, 
questions invariably arise with regard to the technical or 
cost elements being proposed. These questions are provided 
to the offeror who responds with the necessary 
substantiating documentation. The government must exercise 
great care in pre-award discussions with offerors to avoid 
providing technical or cost information about competing 
proposals or appearing to direct technical or cost changes 
in the proposal. Overt technical and cost leveling is a 
prohibited practice, but pre-award discussions frequently 
have the effect of seeing differing proposals move closer 
together in certain technical or cost areas. This can have 
the effect of bringing the higher weighted technical scores 
of competent offerors closer together and cause the lower 
weighted cost score to become more important than 
originally intended. 

4. Cost Realism Scoring 

After the technical and cost analysis have been 
completed, the proposal is evaluated and scored for cost 
realism in accordance with the cost realism scoring plan 
established as a part of the source selection plan. There 
are differing methods for scoring cost realism, with no one 
method being clearly superior. The one similarity all 


methods seem to share is their use of the government cost 


30 


estimate developed during cost analysis as the baseline 


against which the proposal is evaluated and scored. 


D. SOURCE SELECTION 

The source selection decision is made by the source 
selection authority using the evaluation process, 
methodology and techniques established in the source 
selection plan. 

The Source Selection Authority shall consider any 
rankings and ratings, and, if requested, any 
recommendations prepared by evaluation and advisory 
groups. [Ref. l:p. 15-23] 

In practice, the evaluation of a proposal is normally 
performed by the Source Selection Evaluation Board (SSEB) 
in accordance with the evaluation factors and process 
established in the solicitation and source selection plan. 
It is the SSEB that evaluates and scores proposals for cost 
realism. 

The results of the proposal evaluations by the SSEB are 
then provided to the Source Selection Advisory Council 
(SSAC) for review. The SSAC will review the evaluation 
efforts of the SSEB and request clarification or further 
evaluation as required. When all evaluations have been 
satisfactorily completed, the SSAC will compare the 
evaluation results between the competing proposals. When 
the SSAC is finished comparing evaluation results, a 


recommendation of a source for selection is prepared with 


supporting documentation for forwarding to the SSA. 
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The supporting documentation prepared for the selection 
decision shall show the relative differences among 
proposals and their strengths, weaknesses and risks in 
terms of the evaluation factors. The supporting 
documentation shall include the basis and reason for the 
decision. [Ref. 1l:p. 15-23] 


The SSA will review the SSEB proposal evaluations and 
the recommendations of the SSAC for completeness. After 
any additional questions have been answered or 
clarifications received, the SSA will make the source 


selection decision. 


E. SOURCE SELECTION BID PROTEST FOR INADEQUATE COST 
REALISM 
When the source selection decision has been made, the 
government has the responsibility to promptly notify each 
offeror whose proposal was not selected for award. 


The notification shall include- 

(i) The number of offerors solicited 

(ii) The number of proposal received 

(i111) The name and address of each offeror receiving an 
award 

(iv) The items, quantities and unit prices of each 
award (if the number of items or other factors 
makes listing unit prices impracticable, only the 
total contract price need be furnished) and 


(v) In general terms, the reason the offeror's proposal 


was not accepted, unless the price information in 
(iv) above readily reveals the reason. [Ref.1:p. 
15-42] 


At this point, unsuccessful offerors have the right to 
protest the proposed contract award by filing a protest 
with the contracting officer or the Comptroller General. 


belief that the government conducted inadequate cost 
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A 


realism analysis is one reason of many possible reasons for 
filing such a protest. 

Decisions by the Comptroller General in cost realism 
based bid protests have normally hinged on the informed 
judgement rule. The informed judgement rule is a 
requirement for the contracting officer to make an analysis 
of proposed costs that is sufficient for making a 
supportable judgement that the proposed costs are 
realistic. 

If cost and technical analyses have been performed to 
the degree that a source selection decision can be made 
that is both reasonable and consistent with the evaluation 
factors contained in the solicitation, then the Comptroller 
General will most likely consider the decision an informed 
judgment. 


Three points should be kept in mind. First, the 
informed judgment rule is a flexible not a fixed 
standard. What constitutes inadequate cost realism 
analysis in one set of circumstances may be adequate cost 
analysis in another....Second, the procuring agency's 
judgment as to the methods used in developing the 
government's cost estimate and the conclusions 
reached...are entitled to great weight since the 
procurement agencies are in the best position to 
determine the realism of costs....Third, a cost realism 
determination will be considered reasonable if it is 
based on an informed judgment that the proposed costs are 
realistic....An informed judgment results from 
consideration of a number of different factors, l.e. 
independent audits, government's statistics or similar 
contracts for similar services. By considering these and 
other factors, the Contracting Officer may develop and 
document the rational basis necessary for a cost realism 
determination that satisfies the informed judgment rule. 
(Ref. 5:p. 9] 
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F. SUMMARY 

This chapter discussed at when and where cost realism 
issues must be considered during the source selection 
process. First, planning for cost realism occurs when the 
source selection plan is prepared by determining those 
proposal evaluation factors which will be used. Second, 
the realism of proposed costs and technical performance are 
considered during the proposal evaluation process, whether 
or not a formal cost realism analysis takes place. 
Finally, the source selection decision is made based on a 
determination that the selected source is best able to meet 
the government's solicitation requirements, cost, technical 
and other factors considered. The possibililty that a bid 
protest will be filed based on inadequate cost realism 
analysis always exists. However, the likelihood of an 
adverse Comptroller General decision is reduced as long as 
the source selection decision is a supportable judgment 


based on an adequate cost realism analysis. 
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IV. COST REALISM UTILIZATION IN THE DEPARTMENT OF THE NAVY 


A. INTRODUCTION 

This chapter reviews how cost realism is utilized in 
the Department of the Navy. The chapter begins with a 
discussion of cost realism requirements and guidance as 
established by the Office of the Assistant Secretary of the 
Navy, Shipbuilding and Logistics (ASN S&L). The chapter 
then introduces the cost realism practices of the Navy's 
three major system commands: Naval Sea Systems Command 
(NAVSEA), Naval Air Systems Command (NAVAIR) and the Space 


and Naval Warfare Systems Command (SPAWAR). 


B. ASN (S&L) COST REALISM UTILIZATION 
There was no formally established Navy-wide cost 

realism guidance in effect at the time of this study. The 
closest thing to Navy-wide guidance available is the Cost 
Realism Handbook for Assuring More Realistic Contractor 
Cost Proposals promulgated by the Navy Office for 
Acquisition Research: 

The handbook is about source selection cost evaluation 

for competitive, cost reimbursable contracts with 

particular emphasis on assuring more realistic contractor 

cost proposals. It is intended as a practical guide and 

reference for program management personnel and source 

selection personnel. 


The handbook discusses the source selection process and 


the necessity for adequate cost analysis to assure the 
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realism of contractor cost proposals. Techniques and 
procedures are presented for conducting cost realism 
analysis, but they are advisory only and not mandatory for 
source selection decisions. 

The handbook goes into great detail on some of the 
different methods available for scoring cost realism. The 
thrust of the handbook's cost realism scoring methodology 
is the development of an unbiased scoring model against 
which all cost proposals can be compared with a clear 
winner emerging. Where the handbook appears to be 
incomplete is in its treatment of contractor cost proposals 
in comparison to the government's cost estimate. The 
difference between the contractor's cost proposal and the 
government's cost estimate determines the cost realism 
score with no allowance noted for positive or negative 
scores. 

As an example of the handbook's proposed scoring 
method, a contractor proposal that is within plus or minus 
five percent of the government estimate could be awarded a 
maximum score. For a $20 million government estimate, this 
would mean the contractor's proposal can be between $19 and 
$21 million. This presents a problem of contract award. 

If the government awards the contract for any amount less 
than the $20 million government estimate, and the estimate 
was accurate, then there will most likely be a cost growth 


of $1 million in contract completion costs. If the 
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contract is awarded for an amount greater than the 
government estimate, two questions arise. First, how can a 
fair and reasonable price determination be made at an 
amount higher than the government estimate? Second, 
assuming the contractor manages to the contract price, does 
the amount of the contract which exceeds the government 
estimate constitute additional profit and is this a 


violation of existing profit policy? [Ref. 6] 


C. NAVSEA COST REALISM UTILIZATION 
1. Introduction 

NAVSEA uses cost realism analysis to assist the 
source selection decison making process in awarding cost 
reimbursable ship repair overhaul contracts. [Ref. 7] 

NAVSEA obtains ship repair overhauls through what 
could best be described as a two step process. In the 
first step, potential offerors propose against a notional 
specification work package that represents NAVSEA's best 
estimate of the work effort which will be required during 
the ship repair overhaul. Contractor proposals are 
evaluated and scored in accordance with the evaluation 
factors contained in the RFP. Included among these 
evaluation factors are factors for cost realism and cost to 
government. After all proposals have been evaluated and 
scored, the winning offeror is chosen to receive the ship 


repair overhaul award. At this point, step two of the 
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process begins with finalization of the ship repair 
specification package and the commencement of negotiations 
(sole-source) for the terms, conditions and price for the 


actual ship repair overhaul contract. [Ref. 7] 


Ze Cost Realism Analyses Performed 


NAVSEA's evaluation of a proposal begins with the 


selection of a sample of work items from the notional 
specification package for in depth evaluation. This 
selection of sample work items usually represents that ten 
percent of the work package items that have the highest 
value. [Ref. 7] 

Evaluation of the selected work items is based upon 
the evaluator's review of the contractor's back-up data 
used to support the proposed material and manhours of labor 
to complete the work item. If the backup data fully 
supports the contractor's proposed material and labor 
figures in both quantity and quality, no adjustment to the 
proposal is necessary. If the backup data does not 
adequately support the contractor's proposed material or 
labor figures, the proposal is adjusted where required. 

The necessity for adjustment and the level of adjustment 
taken is a subjective determination made by the NAVSEA 
evaluator(s). [Ref. 7] 

Evaluated and adjusted, if necessary, work item 

cost estimates are compared to the contractor's proposed 


work item estimate, and any difference becomes the 
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adjustment factor for that work item. The adjustment 
factors for all work items evaluated are then accumulated, 
and a cumulative adjustment factor is determined. The 
cumulative adjustment factor is then applied to the 
contractor's proposal resulting in an adjusted proposal, 
which also becomes the cost to government estimate. The 
percent difference between the original proposal and the 
adjusted proposal is used to score cost realism. [Ref. 7] 
Cost to government scoring is based upon a ranking 
of all adjusted proposals, from lowest to highest. The 
proposal resulting in the lowest estimated cost - 
government receives the highest score with subsequent 
scores decreasing as cost to government increases. [Ref. 7] 
Scores for cost realism and cost to government are 
weighted and combined with the weighted scores for other 
evaluation factors. The final scores for all offerors are 
then compared, and the highest scoring offeror is selected 


for contract award. [Ref.7] 


D. NAVAIR COST REALISM UTILIZATION 
1. Introduction 
NAVAIR uses cost realism analysis to assist tke 
source selection decision making process in awarding cost 


reimbursable contracts. [ Ref. 8] 
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2. Cost Realism Analysis Performed 


NAVAIR's evaluation of contractor proposals results 
in a thorough, critical review of every proposal received 
in response to an RFP. [Ref. 8] 

NAVAIR evaluators review each element of the WBS 
for completeness, adequacy and supportability. The 
determination of a contractor's capability to perform the 
effort contained in a proposal is made based upon a review 
of the proposal, prior performance of the contractor on 
similar efforts and, in some cases, prior performance of 
other contractors on similar efforts. The objective of 
NAVAIR's critical review is the development of an 
independent estimate of what the product will most probably 
cost the government if manufactured by the contractor. 

This most probable cost becomes the government estimate and 
when compared to the contractor's proposal forms the basis 
for a determination regarding the cost realism of the 
contractor's proposal. [Ref. 8] 

The information developed regarding cost realism of 
the contractor's proposal is provided to the SSAC along 
with all other data generated during the proposal 
evaluation process. The SSAC, after considering evaluation 
reports on all offerors, makes an award recommendation to 


the SSA. [Ref. 8] 
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E. SPAWAR COST REALISM UTILILZATION 
1. Introduction 
SPAWAR uses cost realism analysis to assist the 
source selection decision making process in awarding cost 


reimbursable contracts. [Ref. 9] 


2. Cost Realism Analysis Performed 


Among the Navy's three systems commands, SPAWAR 
alone has developed detailed procedures for the conduct of 
cost realism analysis on contractor proposals. 

The procedure begins with the decision to include a 
requirement for cost realism analysis and scoring in the 
source selection plan. [Ref. 9] Contractors are advised in 
the RFP that cost realism analysis will be performed 
through the inclusion of the clause: 

REALISM OF COST OR PRICE PROPOSALS (M-74 MAY) 


An offeror's proposal is presumed to represent his best 
efforts to respond to the solicitation. Any 
inconsistency, whether real or apparent, between promised 
performance, and cost or price, should be explained in 
the proposal. For example, if the intended use of new 
and innovative production techniques is the basis for an 
abnormally low estimate, the nature of these techniques 
and their impact on cost or price should be explained; 
or, if a corporate policy decision has been made to 
absorb a portion of the estimated cost, that should be 
stated in the proposal. Any significant inconsistency, 
if unexplained, raises a fundamental issue of the 
offeror's understanding of the nature and scope of the 
work required and of his financial ability to perform the 
contract, and may be grounds for rejection of the 
proposal. The burden of proof as to cost credibility 
rests with the offeror. 


Upon receipt of a contractor's proposal, a careful 


and complete evaluation of the proposal is made by the 
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SPAWAR evaluation team. Each element of the WBS is 
examined for completeness, adequacy and supportability. 
Comparisons are made between the contractor's proposed 
costs for material, labor, overhead and known costs for 
similar efforts by this contractor, or other contractors on 
similar efforts. Adjustments are made to the contractor's 
proposal reflecting those changes the SPAWAR evaluators 
believe necessary to make the proposal more realistic. The 
adjusted proposal then becomes the SPAWAR estimate. The 
goal of the SPAWAR evaluation is to determine whether the 
contractor has the capability to perform the required 
effort and what the cost of this effort will be. [Ref. 10] 

To score cost realism, the contractor's cost 
proposal and the SPAWAR cost estimate are plotted on the 
cost realism scoring rule table established for the 
procurement, and a score is awarded. The cost realism 
score becomes a part of the overall score for the proposal, 
as delineated in the source selection plan, and the 
proposal with the highest overall score is awarded the 
contract. [Ref. 10] 

3. The SPAWAR Cost Realism Scoring Rule 

As previously pointed out, the difference between 
the contractor's cost proposal and the SPAWAR cost estimate 
decides the cost realism score as determined by the cost 
realism scoring rule. The principle surrounding the cost 
realism scoring rule is very simple, as demonstrated in 


Figure 4.1. [Ref. 10] 
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The SPAWAR Cost Realism Scoring Rule 


MAX 


SCORE 


MIN 





Source: Adapted from [Ref. 10] 


Figure 4.1 
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The SPAWAR cost estimate (S) determines the 
position on the graph where the maximum score can be 
earned. At predetermined distances from the SPAWAR cost 
estimate are the points B and B' where the minimum score 
will be earned. When the contractor's cost proposal (C) is 
plotted, the score associated with the difference (S-C) is 
determined by the line BAB'. The closer a contractor's 
cost proposal comes to equalling the SPAWAR cost estimate, 
the more the score approaches the maximum. The reverse is 
also true. As the contractor's cost proposal moves away 
from the SPAWAR cost estimate, the score approaches the 
minimum. 

The shape of the line BAB' associated with the cost 
realism scoring rule is dependent upon the particular 
procurement under consideration and the assumptions 
associated with it. Figure 4.2 represents the more common 
shapes used consisting of the triangle, trapezoid, stepped 
and curve. [Ref. 10] 

The triangle is a simple scoring rule to determine 
and plot. Between B or B' and S the score for C is 
proportional to the difference between S and C. 

The trapezoid scoring rule allows a small cost 
range to define the SPAWAR cost estimate where different 
cost proposals can earn the maximum score. This scoring 


rule recognizes the uncertainty which can surround the 
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Common Cost Realism Scoring Rule Shapes 


TRIANGLE PRAPEZOMD 
8) ede ee.) CURVE 


Source: Adapted from [Ref. 10] 


Figure 4.2 
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development of the government's cost estimate and allows 
credit for this uncertainty. 

The stepped scoring rule takes the trapezoid one 
step further by assigning the same score to a range. The 
problems associated with this rule are twofold. First, the 
stepped scoring rule could assign quite different scores to 
two contractors whose cost proposals have similar degrees 
of cost realism (i.e. the two scores straddle a step 
break). Second, the stepped scoring rule could assign the 
same score to two contractors whose cost proposals have 
quite different degrees of cost realism (i.e. the two 
scores are at opposite ends of a step plateau). 

The use of the curve scoring rule, such as the 
normal bell shaped curve, provides an approximation to the 
triangle and trapezoid scoring rules. 

As previously stated, the type and shape of the 
cost realism scoring rule utilized in proposal evaluation 
is largely determined by the nature of the procurement. 

The scoring spread between the maximum score (S) and the 
minimum score (B or B') is usually around 20 percent. The 
belief is that any cost proposal that is more than 20 
percent away from a carefully prepared SPAWAR estimate 
raises fundamental questions about the contractor's 
understanding of the contract requirements and hence the 
cost realism of the proposal [Ref. 10]. Of course, this is 


not to say that a scoring spread in excess of twenty 
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Skewed Cost Realism Scoring Rule Shapes 


SKEWED TRIANGLE SKEWED MRA REZ OD 
Sen Wi Disa EP PED SKEWEDS CURVE 


Source: Adapted from [Ref. 10] 


Figure 4.3 
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percent cannot be established if the nature of the 
procurement requires it. 

Another item considered while determining the type 
and shape of the cost realism scoring rule is the 
distribution of the scoring spread around the SPAWAR cost 
estimate. An even, symmetrical distribution is normally 
provided, but there can be procurements where the 
distribution is skewed as demonstrated in Figure 4.3. 


[Ref. 10] 


F. SUMMARY 

The cost realism approaches taken by the three Navy 
systems commands appear very similar. First, each 
carefully evaluates contractor cost proposals for 
completeness, adquacy and supportability. Then, government 
cost estimates are developed using the contractor's 
proposal as a basis. Other data as considered desireable 
and necessary may be included. Finally, the contractor's 
cost proposal is compared to the government's cost estimate 
to determine the cost realism of the contractor's cost 
proposal. It is in the procedures for scoring cost realism 
that the three systems commands differ the most, with only 
SPAWAR having developed a rather formal and involved 
procedure. The end results for the three systems commands 
is the same, however, with the cost realism score being 
included as a part of the final score determination of a 
contractor's proposal. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. ANSWERS TO THE RESEARCH QUESTIONS 


1 Can the cost realism concept be used to improve the 
source selection decision making process? 


Cost realism requires a source selection decision to 
consider the most likely cost of a contractor's effort. 
The emphasis given to budgetary control issues makes it 
important that a source selection decision consider the 
most likely final cost of a procurement before making an 
award. With cost realism, the procurement program manager 
has a good prediction of what the budgetary requirements 
for the program will be. 

Dis What is cost realism? 

Cost realism is the methodology by which a contractor's 
cost proposal is evaluated during the source selection 
process to determine, with some degree of accuracy and 
fairness, whether or not it is a realistic estimate of the 
actual costs, assuming no changes to the government's 
requirements or the contractor's elements of cost and/or 


technical approach. 


i At what points and under what circumstances may cost 
realism become an issue in the source selection 


process? 


First, during the development of the source selection 


plan when planning for cost realism is first considered. 
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The cost realism methodology to be employed must be capable 
of giving adequate consideration to the various risks 
associated with the procurement while providing a realistic 
estimate of the procurement's actual costs. Second, how 
cost realism will be weighted and scored in the source 
selection plan may become an issue dependent upon the other 
requirements of the procurement and the capabilities of the 
potential offerors. As competing proposals are evaluated 
and considered essentially equal in the other evaluation 
factors, (i.e. technical, management, schedule, cost), cost 
realism may become the determining factor. An inadequate 
cost realism analysis could result in award to an offeror 
whose proposal is not the most advantageous to the 
government and the protest of that award by the 
unsuccessful offeror(s). 


4. How can cost realism best be utilized in the source 
selection process? 


Cost realism can best be utilized as an evaluation 
factor by the SSA in making the source selection decision. 
While performing a cost realism analysis the contractor's 
proposal receives a detailed evaluation that provides an 
estimate of the most likely cost of the contractor's 
proposal. The cost realism score provides an invaluable 
method to compare different proposals and also provides a 
solid basis upon which to defend a source selection 


decision if that should become necessary. 
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B. CONCLUSIONS 


The following conclusions were developed as a result of 


this research effort. 


Cost realism analysis when adequately performed may 
assist the Source Selection Authority in making an 
informed judgement source selection decision. 


As discussed, cost realism can be one of many 
evaluation factors considered by the SSA in making a source 
selection decision. As competing proposals become more 
equal in non-cost related evaluation factors (i.e. 
technical, management, schedule), the cost related 
evaluation factors increase in importance as potentially 
deciding factors. The necessity for an informed judgement 
with regard to the cost related evaluation factors 
increases the probability that a source selection decision 


will be supportable and defensible. 


An adequate cost realism analysis requires a thorough and 
complete review of all aspects of a contractor's proposal 


to develop a costing baseline for the government 
estimate. 


It is in comparison against the evaluated government 
estimate that the cost realism of a contractor's proposal 
is determined. The government estimate is developed 
through a careful and complete evaluation of the elements 
of cost (material, labor, overhead), which make up the 


contractor's cost proposal. 


ot 


Cost realism is an excellent forecaster of what a program 
will most likely cost; however, cost realism is not a 
guarantee of what the program's final cost will be. 


The government estimate of a contractor's most likely 
cost is still just an estimate. It is subject to the risks 
inherent to most cost reimbursable procurements: schedule, 
risk, design risk and production risk. If all risk could 
be eliminated from a program, there would be no need for a 
cost reimbursable contract award. Bids could be taken from 
potential offerors, and the award would be made to the 
lowest cost bidder. Since risk exists, there is the 
distinct possibility that a program will experience cost 


growth changes to the estimated final cost. 


C. RECOMMENDATIONS 
The following recommendations are relevant from this 


research effort. 


A standard methodology for determining the cost realism 
of a contractor's proposal should be established and 
required for all cost reimbursable contract awards. 


The FAR and service implementing directives and 
instructions should be revised to give specific guidance on 
determining the cost realism of a contractor's proposal. 
Developing standards for cost realism analysis and 
requiring a cost realism determination for contractor 
proposals will ensure that source selection decisions are 


informed judgments. 
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Cost realism should be used for any procurement where the 
contractor is required to provide cost and pricing data. 


Sole source negotiations for a cost reimbursable 
contract could use cost realism analysis to assist in 
determining a fair and reasonable price for the 


procurement. 


D. RECOMMENDATIONS FOR FURTHER RESEARCH 

A study should be conducted to evaluate the optimum 
characteristics of cost realism scoring rules and the 
differing procurement circumstances under which their use 
can be maximized. 

A study should be conducted to determine how cost 
realism is evaluated in other government agencies. 

A study should be conducted to review the effect cost 
realism has on how contractors develop and present 


competitive cost proposals. 
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